Skip to content

fix(plugin): align command descriptions across harnesses and test alignment - #55

Merged
Facundo Santiago (santiagxf) merged 1 commit into
fix/plugin-yaml-frontmatterfrom
fix/plugin-adapter-drift
Nov 16, 2026
Merged

Facundo Santiago (santiagxf) merged 1 commit into
fix/plugin-yaml-frontmatterfrom
fix/plugin-adapter-drift

Conversation

@santiagxf

Copy link
Copy Markdown
Member

Stacked on #50. Targets fix/plugin-yaml-frontmatter, not main, because it extends the checker that PR adds.

The problem

Every command's description is written twice: once in commands/<name>.md for the Markdown hosts, once in commands/aigateway/<name>.toml for Gemini. Nothing ties the copies together, so they drift.

feedback has drifted, and it is user-visible today:

Source Describes
commands/aigateway/feedback.toml "Package a selected session into a local feedback ZIP, with mandatory anonymization and review for GitHub issues."
commands/feedback.md "Collect session feedback… Process and anonymize a session summary, returning a clean file ready for sharing."
skills/feedback-ai-gateway/SKILL.md (v2.0.0) summary only, scrub identifiers, return a clean file

The skill moved to returning an anonymized summary and the Markdown followed. The Gemini adapter did not. Gemini users are shown a description of a feature that no longer exists.

The stale description was the visible half. The adapter's prompt still drove the entire v1 flow — consent for a session export, produce a ZIP, publish to a GitHub issue. The skill it points at now rejects full transcripts and states that sharing decisions belong to the user. The adapter was instructing Gemini to do what the skill refuses.

What changed

commands/aigateway/feedback.toml rewritten to mirror commands/feedback.md: summary file in, anonymized file out, no ZIP, no sharing.

The checker now compares the pair. Each .toml description must equal the description of the command it mirrors. The other five already match, and every skills/.../SKILL.md path referenced by an adapter resolves.

Skill directories no longer carry development files. skills/feedback-ai-gateway/tests/{evals.json,test_feedback.py} ship to everyone who installs the plugin. They move to tests/feedback/, and the checker fails if test or eval files reappear under skills/. The test's import path is updated; all 8 tests still pass.

Verification

  • Run against the previous tree, the checker reports exactly the three problems this PR fixes: the adapter drift and the two bundled test files.
  • Run against this branch: passes, with and without PyYAML.
  • Negative tests confirmed to fail: a reworded .toml description, and a new fixture file planted under skills/.
  • python3 -m unittest discover -s tests/feedback → 8 tests, OK.
  • The adapter check needs tomllib (Python 3.11+) and is skipped below that, the same way the strict YAML parse is skipped without PyYAML. CI has both.

Provenance

Both checks are adapted from tests/test_plugin.py in #38. That PR asserts the same two properties; this takes the ideas without the rest of the branch, which is on an orphaned base and conflicting. A third idea there, a 1024-character cap on descriptions, is not included — the longest description in the plugin is 365 characters, so the check could not fire.

🤖 Generated with Claude Code

The feedback command's description is written twice, once in
commands/feedback.md for the Markdown hosts and once in
commands/aigateway/feedback.toml for Gemini, with nothing tying the copies
together. So they drifted. The skill moved to returning an anonymized
summary, the Markdown followed, and the Gemini adapter went on telling
users it packages a local ZIP for GitHub issues -- a flow the current skill
explicitly refuses.

The stale description was the visible half. The adapter's prompt still drove
the whole v1 flow: consent for a session export, a ZIP, GitHub publication.
The skill it points at now rejects full transcripts and says sharing
decisions belong to the user, so the adapter was instructing Gemini to do
what the skill forbids. Rewrite it to mirror the Markdown command.

Check the pair from now on: each .toml description must equal the
description of the command it mirrors. The other five already matched, and
every skill path the adapters reference resolves.

Also move the feedback skill's tests out of the skill directory. Whatever
sits in skills/ is downloaded by everyone who installs the plugin, and
evals.json and test_feedback.py are development files. They move to
tests/feedback/, and the check fails if anything like that reappears.

Both checks were run against the previous tree and report exactly these
three problems.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@santiagxf Facundo Santiago (santiagxf) changed the title Stop the Gemini adapters drifting from the commands they mirror fix(plugin): align command descriptions across harnesses and test alignment Nov 16, 2026
@santiagxf
Facundo Santiago (santiagxf) merged commit 291d0b0 into main Nov 16, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants